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A  WEAK  MEAN  SQUARE  ERROR  TEST 
FOR  STOCHASTIC  RESTRICTIONS  IN  REGRESSION 


T.A.  Yancey,  M.E.  Bock  and  G.G.  Judge 


The  weak  mean  square  error  results  of  Wallace,  relating  to  tests  for 
linear  restrictions  in  regression,  are  extended  to  include  tests  for  sto- 
chastic linear  restrictions. 


1 .   Introduction 

In  a  1968  paper,  Toro-Vizcarrondo  and  Wallace  [6]  showed  that  the  non- 
central  F  distribution  with  a  non-centrality  parameter  of  one-half  could  be 
used  to  test  the  hypothesis  that  a  linear  equality  restricted  estimator,  $, 
is  better  than  an  unrestricted  least  squares  estimator,  b,  where  better 
refers  to  the  generalized  mean  square  error  (MSE)  criterion.  This  means 
for  the  estimators  3  and  b  that  MSE(^|)  <  MSE(£b)  for  every  ^  j^  o.   In  a 
companion  paper,  the  same  authors  [8]  present  tables  of  critical  points, 
review  the  test  procedure  and  give  some  examples  of  how  the  test  may  be 
used.   In  a  1970  paper,  Yancey,  Judge  and  Bock  [9]  present  a  test  for  deter- 
mining the  conditions  under  which  stochastic  linear  prior  information,  which 
may  be  incorrect  on  the  average,  may  improve  the  parameter  estimates  for 
the  linear  hypothesis  model  over  conventional  sample  information  estimates 
in  the  sense  of  having  the  same  or  smaller  mean  square  errors  for  all  esti- 
mates.  In  a  1971  paper  in  this  Journal,  Wallace  [7]  develops  tests  of 
betterness  of  the  restricted  estimator  over  the  unrestricted  least  squares 
estimator  where  betterness  is  defined  in  terms  of  average  squared  Euclidean 
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distance  (weak  mean  square  error  (WMSE)),  such  that  in  estimation  for  the 
vector  6, 

(1.1)  E(§-e3'(e-e)  =  tr  (Egg  +  (Bias  f)  (Bias  §)•)  =  i:MSE(e^). 

In  a  companion  paper,  Goodnight  and  Wallace  [1]  develop  tables  to  be  used 
with  the  weak  mean  square  error  tests  for  linear  restrictions  in  regression 

models. 

In  this  note,  we  extend  the  weak  mean  square  error  results  of  Wallace 
[7]  to  include  stochastic  linear  restrictions  and  show  that  the  test  sta- 
tistic for  the  hypothesis 

(1.2)  £(§-§)' (§-6)  <  E(b-§)'(b-§), 

where  B  is  the  stochastic  restricted  estimator  and  b  is  the  unrestricted 
least  squares  estimator,  has  a  non-central  F  distribution  with  a  non-cen- 
trality  parameter  of  J/2,  where  J  is  the  number  of  stochastic  restrictions. 

2.  The  Statistical  Model 


Consider  the  linear  statistical  model 


C2.1)     y  =  XB  +  u, 

(2.2)-''    E(u)  =  0    and    E(uu')  =  a^I, 

where  j^  is  a  (T  x  i)  vector  of  observations,  X  is  a  (T  x  K)  matrix  of  non- 


-    Although  we  make  use  of  the  classical  stochastic  assumptions  regard- 
ing the  disturbances,  the  results  of  this  paper  are  also  valid  for  the  more 
general  specification  0^6,  where  9  is  some  symmetric  positive  definite  matrix. 
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stochastic  variables  of  rank  K,  u  is  a  (T  x   1)  vector  of  unobservable  random 
normal  variables,  I  is  a  unit  matrix  of  order  T,  and  B  is  a  (K  x  I)  vector 
of  unknown  parameters. 

Assume,  in  line  with  the  work  of  Theil  and  Goldberger  [5],  Theil  [4], 
Kakwani  [3]  and  others,  the  following  stochastic  prior  information  about  3: 

(2.3)  r  =  R6  +  V, 

(2.4)  E(v)  =  6,   E(vv')  =  a^J^^   and   ECuv')  =  0, 

where  R  is  a  (J  x  K)  known  matrix  of  rank  J,  r  is  a  (J  x  i)  vector  of  prior 
estimates  of  R6,  v  is  a  (J  x  1)  random  vector  of  unobservable  normal  random 

variables  with  mean  vector  6,   representing  the  uncertainty  about  the  possibly 

2 
biased  prior  information  r,  and  o^     is  a  known  non- singular  matrix.  The 

2  2 
ratio  m  =  o  /o     is  also  assumed  known.   In  the  discussion  to  follow,  we 

define  Q  =   mJ^„. 

0 

Under  these  specifications,  the  stochastic  restricted  estimator  (Theil 
and  Goldberger  [5]  and  Theil  [4]),  which  makes  use  of  the  Aitken  estimator 
to  combine  the  sample  and  stochastic  prior  information,  is 

(2.5)  §  =  [or~^X'X  +  a'^R'n"-^R]~-^[a"^X'j'  +  a"^R'Q"-^:r] 

=   [X'X  +  R'P."-^R]"-^[X'y  +  R'f2"-^r], 

and  the  variance- covariance  of  B  is 

(2.6)  E(3-EB)(3-EB)'   =  0^[X'X  +  R'fl'-^R]"-^  =  Zgg. 

-  -  -  -  ^p 

The  unrestricted  least  squares  estimator,  using  only  sample  information, 
is 

(2.7)  b  =   (X'X)'^X'y, 


where  the  variance-covariance  of  b  is 

(2.8)  ECb-Eb)(b-Eb)'   =  a^CX'X)""^  =  Z,  ^ . 

-  -  -  -  Ob 

If  we  make  use  of  the  mean  square  error  criterion,  and  the  correspond- 
ing  expected  mean  square  error  matrix  of  the  estimator  3,  then  we  may  write 
the  risk  matrix  as 

(2.9)  E(6-e)(e-B)'   =  MSE^s  =  oV-"-  +  W"-^Rf2'-^66'fi"-^R'W""^, 

-  -   -  -  tip 


where 


W  =   (X'X  +  R'J^"'^R). 


The  difference  between  all  linear  combinations  of  the  expected  loss 
for  the  stochastic  restricted  estimator  B  and  the  unrestricted  least  squares 
estimator  b  is 

(2.10).   A^  =  rE(b-§)(b-§)'^  -  rE(0-$)(0-§)'fc  =  rMSEj^^£  -  i'MSEgg^ 

=  A' (X'X  +  R'J^~-^R)"-^R'i"i"-^[a^(R(X'X)"-^R'  +  «)  -  66' ]  [R^"'^  (X'X 

+  R'n"'^R)"-^]A, 

which  must  be  greater  than  or  equal  to  zero  if  the  stochastically  restricted 
estimator  is  to  be  superior  using  the  MSE  criterion. 

In  order  to  test  the  compatibility  of  the  stochastic  prior  a.id  sample 
information  using  the  mean  square  error  criterion  Yancey,  Judge  and  Bock  [9], 
following  the  reasoning  of  Toro-Vizcarrondo  and  Wallace  [6],  developed  a 
test  statistic  which  has  a  non-central  F(J,T-K,A)  distribution  and  showed 
that  every  linear  combination  of  the  elements  of  the  mean  square  error 
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matrix  of  §  is  as  small  or  smaller  than  the  same  linear  combinations  of  the 
elements  of  the  mean  square  error  matrix  of  b  if  and  only  if  the  non-cen- 
trality  parameter  A  is  <  y  , 


5.  The  Weak  Mean  Square  Error  Criterion  and  Test 

As  noted  by  Wallace  [7],  to  have  the  mean  square  error  of  every  combin- 
ation of  I  better  than  the  corresponding  linear  combination  of  b  is  quite  a 
strong  requirement.  Wallace  [7]  suggests  what  he  terms  a  weak  mean  square 
error  (WMSE)  criterion  as  a  reasonable  but  weaker  requirement.  For  vector 
estimation,  this  criterion  may  be  stated  in  general  as 

(3.1)  ^2  -  E(e-e)'Ce-e)  =  tr (Egg  +  (Bias  §) (Bias  §)')  =  EMSE(6.), 

i 

where  A_  is  the  expected  squared  Euclidean  distance  from  9  to  G. 

The  WMSE  criterion  for  the  stochastic  linear  restriction  estimator  may 
be  written  as 

(3.2)  WMSE  =  tr  E(b-§)(b-§)'  -  tr  E(§-§)  (§-§)  • 

=  tr(X'X  +  R'rr-^R)"-^RJ^"-^[a^(R(X'X)"-^R'  +  fi)  -  66 ' ]  [J^"^R(X'X 

+  R'fi'V'"^]. 

In  order  for  the  stochastic  restricted  estimator  to  be  superior  via 
this  criterion,  equation  (3.2)  must  be  equal  to  or  greater  than  zero. 
From  (5.2),  we  can  write 

(3.3)  A2  =  r(X'X+  R'fi"-^R)"-^R'n"-^[a^(R(X'X)~-^R'  +  fi)  -  66 '  ]n"'^R(X'X 

+  R'ft'-^R)"'^^ 
=  £'A'BAJ?., 


which  is  positive  semi-definite  for  all  non-zero  (J  x  1)  vectors  i   if  and 
only  if 

(3.4)  B  =   [a^(R(X'X)"-^R'  +  fi)  -  66'] 

2/ 
is  positive  semi -definite.— 

Hence , 

(3.5)  tr  B  =  tr[a^(R(X'X)"-^R'  +  ^)    -  66']     >     0. 
The 

(3.6)  tr[a^(R(X'X)"-^R'   +  f2)   -  55'] 
can  be  written  as 

(3.7)  tr[o-I   -  66'(R(X'X)"-^R'    +  fi)"'^]  (R(X' X)"-^R'    +  fi) , 

where  (R(X'X)~  R'  +  il)   is  positive  definite.  Since  the  characteristic  roots 
of  a  positive  definite  matrix  are  ail  positive,  from  (3,5)  the 

(3.8)  tr[a^I  -  66"  (R(X'X)'-^R'  +  fi)~-^]  (R(X'X)"-^R'  +  ft) 

is  equal  to 

n 

(3.9)  I  i>.  r-y'^-      >   0 
j^^j  jCl)  i   = 


2/  _1   _1 

—  This  can  be  seen  since  A  =  (X'X  +  R'fi  R)   is  positive  definite  by 

assumption,  and  we  can  let  the  s,  a  non-negative  (J  x  1)  vector  in  the 
quadratic  form,  in  s '  Bs  >  0  equal  Ail.  Given  either  s  or  i,   we  can  find 
i  =  As  or  s  =  A'^a,     Clearly,  s  =  0,  a  null  vector.  If  and  only  if  il  =  0. 
Thus,  C  is  positive  semi-definite  if  and  only  if  A'BA  is  positive  semi- 
definite  since  V ABM  =   s'Bs  >  0  for  all  s  and  I   not  identically  zero  and 
A'AB.AK.  =  sBs  =  0  for  some  s  and  Z  ^  0.      Furthermore,  if  a  matrix  is  posi- 
tive semi-definite,  its  trace  is  non-negative  (Graybill  [2,  p.  318]),  and 
we  have  tr  A'BA  >  0  if  and  only  if  tr  B  >  0. 
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for  some  ordering  of  if).  ,  where  (}).  are  characteristic  roots  of  (R(X'X)  R'  +  fj) 
and  ii.   the  characteristic  roots  of  [a^I  -.  66' (R(X'X)"-^R'  +  Q)""^]  (Graybill 
[2,  p.  233]). 

Since  the  <().  are  all  positive  and  since  E^., ..<!>.  >  0  for  all  sets  of 

-1  2 

positive  definite  and  semi -definite  matrices  (RCX'X)  R'  +  fi)  and  [o  I 

-  66' (RCX'X)  R'  +  f2)  ]  respectively,  the  'i'-r--^   are  all  non-negative. 

Furthermore,  the  characteristic  roots  of  positive  semi-definite  matrices 

2  -1      -1 

are  non-negative,  [a  I  -  66'(R(X'X)  R'  +  fi)   ]  is  positive  semi-definite, 

and  tr[a^l  -  66' (R(X'X)'-^R'  +  ^)~^]   >   0  which  implies 

(3.10)  J0^  -  6'(R(X'X)"-'-R'  ^  Qy^6     >     0, 
or 

(3.11)  y  >  -iy  6'(R(X'X)"-^R'  +  fi)6  =  X. 
'^  "  20^  " 

Hence,  if  the  non-centrality  parameter  A  of  the  distribution  of  y   is  less 

J 
than  or  equal  to  j,   the  estimator  3  is  better  than  b  when  judged  by  the 

weak  MSE  criterion.  Similarly,  X  >  ^  if  and  only  if  E(b-B)'Cb-B)  < 
E(|-§) •(§-§). 

Given  this  result,  one  can  test  the  hypothesis  that  the  use  jf  sto- 
chastic linear  restrictions  improves  the  estimator  in  a  weak  MSE  sense 
over  its  unrestricted  least  squares  counterpart,  by  testing  the  hypothesis 
X  <  y  against  the  alternative  X  >  -j  and  by  using  the  tables  developed  by 
Goodnight  and  Wallace  [1],  noting  whether  y  >  T^Cy.J.T-K)  at  the  a  test 
level. 


4.   Concluding  Remarks 

A  test  for  stochastic  restrictions  in  regression  which  recognizes  the 
trade-off  between  bias  and  variance  has  been  developed.   It  should  be  noted 
in  closing  that  if  we  remove  the  assumption  that  m,  the  ratio  of  the  vari- 
ances of  the  prior  restriction  and  sample  errors,  is  known,  statistics  are 
created  whose  distributions  are  unknown  and  untabled.  However,  for  the 
statistical  model  of  this  paper,  preliminary  sampling  experiments  indicate 
that  the  non-central  F  distribution  is  a  reasonable  approximation  to  the 
empirical  distribution  when  m   is  replaced  by  an  estims^te,  m  =  ^o~/a   .  Fin- 
ally, let  us  note  that  as  in  the  equality  restricted  estimator  case,  the 
properties  of  the  preliminary  test  estimator, 

which  are  based  on  the  preliminary  test  about  X,  where 


and 


for 


^(0,c)^^^  "  °    ^""^    ^(c,«)^^^  "  "'    "^^^^  Y  >  c. 


00 

/  fCY)dY  =  a, 
c 

are  yet  to  be  determined.  The  authors  are  currently  working  on  the  samp- 
ling properties  of  this  and  related  preliminary  test  estimators. 


University  of  Illinois 
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